Protein optimization is a fundamental biological task aimed at enhancing theperformance of proteins by modifying their sequences. Computational methodsprimarily rely on evolutionary information (EI) encoded by protein languagemodels (PLMs) to predict fitness landscape for optimization. However, thesemethods suffer from a few limitations. To address these limitations, we introduce DenoisingProtein Language Models (DePLM), a novel approach that refines the evolutionaryinformation embodied in PLMs for improved protein optimization. Our approachinvolves denoising this EI in PLMs through a diffusion process conducted in therank space of property values, thereby enhancing model generalization and ensuringdataset-agnostic learning.

We report a flexible language-model based deep learning strategy, applied here to solve complex forward and inverse problems in protein modeling, based on an attention neural network that integrates transformer and graph convolutional architectures in a causal multi-headed graph mechanism, to realize a generative pretrained model. The model is applied to predict secondary structure content (per-residue level and overall content), protein solubility, and sequencing tasks. Further trained on inverse tasks, the model is rendered capable of designing proteins with these properties as target features. The model is formulated as a general framework, completely prompt-based, and can be adapted for a variety of downstream tasks. We find that adding additional tasks yields emergent synergies that the model exploits in improving overall performance, beyond what would be possible by training a model on each dataset alone. Case studies are presented to validate the method, yielding protein designs specifically focused on structural proteins, but also exploring the applicability in the design of soluble, antimicrobial biomaterials. While our model is trained to ultimately perform 8 distinct tasks, with available datasets it can be extended to solve additional problems. In a broader sense, this work illustrates a form of multiscale modeling that relates a set of ultimate building blocks (here, byte-level utf8 characters that define the nature of the physical system at hand) to complex output. This materiomic scheme captures complex emergent relationships between universal building block and resulting properties via a synergizing learning capacity to express a set of potentialities embedded in the knowledge used in training, via the interplay of universality and diversity.

OpenAI's DALL.E 2 has been making it big with text-to-image models that easily generate pictures from textual descriptions. Earlier this week, two biotech labs--Generate Biomedicines and David Baker's Group--relied on generative AI, particularly diffusion models, to come up with new protein structures and, eventually, better drugs. Boston-based therapeutics company Generate Biomedicines announced a programme called Chroma which, according to the company, is the "DALL-E 2 of biology". Similarly, biologist David Baker's team from the University of Washington has also come up with RoseTTAFoldDiffusion. The model can build accurate designs for new proteins that can be brought to life in the lab.

Today, two labs separately announced programs that use diffusion models to generate designs for novel proteins with more precision than ever before. Generate Biomedicines, a Boston-based startup, revealed a program called Chroma, which the company describes as the "DALL-E 2 of biology." At the same time, a team at the University of Washington led by biologist David Baker has built a similar program called RoseTTAFold Diffusion. In a preprint paper posted online today, Baker and his colleagues show that their model can generate precise designs for novel proteins that can then be brought to life in the lab. "We're generating proteins with really no similarity to existing ones," says Brian Trippe, one of the co-developers of RoseTTAFold. These protein generators can be directed to produce designs for proteins with specific properties, such as shape or size or function.

Artificial-intelligence tools are helping to scientists to come up with proteins that are shaped unlike anything in nature.Credit: Ian C Haydon/UW Institute for Protein Design In June, South Korean regulators authorized the first-ever medicine, a COVID vaccine, to be made from a novel protein designed by humans. The vaccine is based on a spherical protein'nanoparticle' that was created by researchers nearly a decade ago, through a labour-intensive trial-and error-process1. Now, thanks to gargantuan advances in artificial intelligence (AI), a team led by David Baker, a biochemist at the University of Washington (UW) in Seattle, reports in Science2,3 that it can design such molecules in seconds instead of months. 'The entire protein universe': AI predicts shape of nearly every known protein Such efforts are a part of a scientific sea change, as AI tools such as DeepMind's protein-structure-prediction software AlphaFold are embraced by life scientists. In July, DeepMind revealed that the latest version of AlphaFold had predicted structures for every protein known to science.

Natural languages and proteins are actually similar in structure. Amino acids arrange themselves in a multitude of combinations to form structures that have specific functions in the living organism -- similar to the way words form sentences in different combinations that express certain facts. In recent years, numerous approaches have therefore been developed to use principles and processes that control the computer-assisted processing of natural language in protein research. "Natural language processing has made extraordinary progress thanks to new AI technologies. Today, models of language processing enable machines not only to understand meaningful sentences but also to generate them themselves. Such a model was the starting point of our research. With detailed information concerning about 50 million sequences of natural proteins, my colleague Noelia Ferruz trained the model and enabled it to generate protein sequences on its own. It now understands the language of proteins and can use it creatively. We have found that these creative designs follow the basic principles of natural proteins," says Prof. Dr. Birte Höcker, Head of the Protein Design Group at the University of Bayreuth.

In the world plagued by the emergence of new diseases, it is essential that we accelerate the drug design process to develop new therapeutics against them. In recent years, deep learning-based methods have shown some success in ligand-based drug design. Yet, these methods face the problem of data scarcity while designing drugs against a novel target. In this work, the potential of deep learning and molecular modeling approaches was leveraged to develop a drug design pipeline, which can be useful for cases where there is limited or no availability of target-specific ligand datasets. Inhibitors of the homologues of the target protein were screened at the active site of the target protein to create an initial target-specific dataset.